ISSUES&ANSWERS 



REL 2011-No. 102 




How student 
and school 
characteristics are 
associated with 
performance 
on the Maine 
High School 
Assessment 





NATIONAL CENTER for 

EDUCATION EVALUATION 
and REGIONAL ASSISTANCE 




Institute of Education Sciences 





ISSUES ANSWERS 



REL 2011-No. 102 




REL 



NORTHEAST 
6? ISLANDS 



Regional Educational Laboratory 
At Education Development 
Center, Inc. 



How student and school characteristics 
are associated with performance on 
the Maine High School Assessment 

Summary 



February 2011 



Prepared by 
Craig D. Hoyle 

Education Development Center, Inc. 

Laura M. O'Dwyer 
Boston College 

Quincy Chang 

Education Development Center, Inc. 







NATIONAL CENTER for 

EDUCATION EVALUATION 
and REGIONAL ASSISTANCE 



Institute of Education Sciences 





NORTHEAST 
S> ISLANDS 



Regional Educational Laboratory 
At Education Development 
Center, Inc. 



Issues & Answers is an ongoing series of reports from short-term Fast Response Projects conducted by the regional educa- 
tional laboratories on current education issues of importance at local, state, and regional levels. Fast Response Project topics 
change to reflect new issues, as identified through lab outreach and requests for assistance from policymakers and educa- 
tors at state and local levels and from communities, businesses, parents, families, and youth. All Issues & Answers reports 
meet Institute of Education Sciences standards for scientifically valid research. 

February 2011 

This report was prepared for the Institute of Education Sciences (IES) under Contract ED-06-CO-0025 by Regional Educa- 
tional Laboratory Northeast and Islands administered by Education Development Center, Inc.. The content of the publica- 
tion does not necessarily reflect the views or policies of IES or the U.S. Department of Education nor does mention of trade 
names, commercial products, or organizations imply endorsement by the U.S. Government. 

This report is in the public domain. While permission to reprint this publication is not necessary, it should be cited as: 

Hoyle, C.D., O’Dwyer, L.M., and Chang, Q. 2011. How student and school characteristics are associated with performance 
on the Maine High School Assessment. (Issues & Answers Report, REL 2011-No. 102). Washington, DC: U.S. Department 
of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional 
Educational Laboratory Northeast and Islands. Retrieved from http://ies.ed.gov/ncee/edlabs. 

This report is available on the regional educational laboratory web site at http://ies.ed.gov/ncee/edlabs. 



Summary rel 2011-No. 102 

How student and school characteristics 
are associated with performance on 
the Maine High School Assessment 



Using multilevel regression models to 
examine how student characteristics, 
student prior achievement measures, and 
school characteristics are associated with 
performance on the Maine High School 
Assessment, the study finds statistically 
significant relationships between several 
of these variables and assessment scores 
in reading, writing, math, and science. 

The Maine Department of Education wanted 
to use longitudinal data from its data system 
to better understand whether and how student 
and school characteristics are associated with 
student performance on the state-mandated 
Maine High School Assessment (MHSA). It 
was particularly interested in understanding 
the factors associated with changes in test 
scores between the beginning of grade 10 and 
the end of grade 11. The MHSA, which com- 
prises the College Board’s SAT tests in criti- 
cal reading, writing, and math and a science 
assessment, is administered in the spring of 
grade 11 to determine whether Maine high 
schools have made adequate yearly progress. 

The following research question guided this 
study: 

• How are student characteristics, student 
prior achievement measures, and school 
characteristics associated with students’ 



grade 11 Maine High School Assessment 
scores in reading, writing, math, and 
science? 

The Maine Department of Education pro- 
vided data on MHSA scores, characteristics, 
and prior achievement measures for all grade 
11 students in Maine in 2007/08. The prior 
achievement measures examined were grade 8 
Maine Educational Assessment (MEA) scores 
in reading, writing, math, and science and 
grade 10 Preliminary SAT/National Merit 
Scholarship Qualifying Test (PSAT) verbal, 
math, and writing scores. 

Data on school characteristics were either pro- 
vided directly by the Maine Department of Edu- 
cation or gathered from publicly available data. 
The data consisted of percentage of racial/ethnic 
minority students, economically disadvantaged 
students, students in special education, English 
language learner students, and students who 
drop out; student-teacher ratio; mean years of 
teaching; cohort size; school location; whether 
the school made adequate yearly progress in 
reading and math in 2007; and whether the 
school was classified as Title I in 2007. 

Multilevel regression modeling, which allows 
individual and group characteristics to be in- 
cluded in models of individual outcomes, was 
used to analyze the associations. 
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In the primary model used to address the 
research question, statistically significant re- 
lationships were found between some student 
and school characteristics and grade 8 MEA 
and grade 10 PSAT scores and grade 11 MHSA 
scores in reading, writing, math, and science, 
holding all other variables constant. 

The major findings related to student charac- 
teristics as predictors of MHSA domain scores 
are: 

• Gender was a significant predictor in all 
four MHSA domains. Male students were 
predicted to have significantly higher 
MHSA scores in reading (-0.052 stan- 
dard deviation), math (-0.088 standard 
deviation), and science (-0.183 standard 
deviation) than female students. Female 
students were predicted to have signifi- 
cantly higher MHSA writing scores (0.060 
standard deviation) than male students. 

• Economically disadvantaged students 
were predicted to have significantly lower 
MHSA scores in reading (-0.035 stan- 
dard deviation), writing (-0.061 standard 
deviation), and math (-0.039 standard 
deviation) than non-economically disad- 
vantaged students. 

• Students in special education were pre- 
dicted to have significantly lower MHSA 
scores in reading (-0.065 standard devia- 
tion), writing (-0.117 standard deviation), 
and math (-0.105 standard deviation) 
than general education students. 

• English language learner students were 
predicted to have significantly lower 
MHSA scores in reading (-0.061 standard 



deviation), writing (-0.102 standard devia- 
tion), and science (-0.100 standard devia- 
tion) than non-English language learner 
students. 

The major findings related to prior achieve- 
ment on the MEA and PSAT as predictors of 
MHSA domain scores are: 

• With three exceptions, grade 8 MEA and 
grade 10 PSAT scores were significantly re- 
lated to MHSA scores. Within each MHSA 
domain, the most recent measure of prior 
achievement with the closest match in 
content was the strongest predictor: 

• The strongest predictors of MHSA 
reading scores were grade 10 PSAT 
verbal and writing scores. 

• The strongest predictor of MHSA writ- 
ing scores was PSAT writing scores. 

• The strongest predictor of MHSA math 
scores was grade 10 PSAT math scores. 

• The strongest predictor of MHSA sci- 
ence scores was grade 8 MEA science 
scores. 

The major findings related to school charac- 
teristics as predictors of MHSA domain scores 
are: 

• Students in schools with higher percent- 
ages of grade 11 economically disadvan- 
taged students were predicted to have 
significantly lower MHSA reading scores. 

• Students in schools with higher per- 
centages of grade 11 students in special 
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education were predicted to have signifi- 
cantly lower MHSA science scores. 

• Students in schools with higher percent- 
ages of students in cohort who drop out 
were predicted to have significantly higher 
MHSA reading scores. 

• Students in schools with higher student- 
teacher ratios were predicted to have sig- 
nificantly lower MHSA scores in reading, 
writing, and math. 

• Students in schools with larger grade 11 
cohorts were predicted to have signifi- 
cantly higher MHSA math scores. 

• Compared with students in rural schools, 
students in suburban schools were pre- 
dicted to have significantly higher MHSA 
writing scores. 

• Students in schools that made adequate 
yearly progress in reading in 2007 were 
predicted to have significantly higher 
MHSA writing and math scores. 

Because the PSAT is a preliminary assess- 
ment for the SAT, which is used for the 
reading, writing, and mathematics domains 
of the MHSA and was administered in the Note 

beginning of grade 10, the strong relation- 
ship between PSAT scores and MHSA scores 
could influence both the magnitude of the 
multilevel regression coefficients associ- 
ated with the other variables and the overall 
interpretation of the model. To the extent that 
student characteristics, grade 8 MEA scores, 
and school characteristics relate to grade 10 
PSAT scores as well as grade 11 MHSA scores, 
the regression coefficients for the remaining 



predictor variables may be attenuated relative 
to a model that does not include PSAT scores. 
Indeed, when PSAT scores are included in the 
model, the coefficients for the non-PSAT vari- 
ables can, to some extent, be interpreted as 
the relationship between those variables and 
the change in achievement between grades 
10 and 11. This is particularly the case with 
respect to the MHSA reading, writing, and 
math scores. 

The findings from this study will inform 
further efforts by the Maine Department 
of Education to understand the association 
between student and school factors and high 
school achievement. To the extent that MHSA 
scores, student characteristics, PSAT scores, 
school characteristics, and unmeasured factors 
remain the same, results of the current study 
can be generalized to other cohorts of Maine 
students who complete all four sections of the 
MHSA in grade 11. 1 Results from this study 
could also be used to identify students and 
schools likely to benefit most from additional 
assistance. Because this is a descriptive rather 
than an experimental study, the findings are 
not sufficient to support causal inferences. 
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1. In 2009, the grade 8 assessment in Maine 
changed from the MEA, taken by the 2007/08 
cohort of students, to the New England Com- 
mon Assessment Program (NECAP). Despite the 
potential differences between the MEA and the 
NECAP, grade 8 MEA scores represent measures 
of achievement prior to students’ entry to their 
respective high schools and are therefore still of 
interest to the Maine Department of Education 
as it examines how information in its evolving 
longitudinal data system can be used to improve 
the education of students in Maine. 



